Recurrent neural network models (CovRNN) for predicting outcomes of patients with COVID-19 on admission to hospital: model development and validation using electronic health record data.

BACKGROUND: Predicting outcomes of patients with COVID-19 at an early stage is crucial for optimised clinical care and resource management, especially during a pandemic. Although multiple machine learning models have been proposed to address this issue, because of their requirements for extensive data preprocessing and feature engineering, they have not been validated or implemented outside of their original study site. Therefore, we aimed to develop accurate and transferrable predictive models of outcomes on hospital admission for patients with COVID-19. METHODS: In this study, we developed recurrent neural network-based models (CovRNN) to predict the outcomes of patients with COVID-19 by use of available electronic health record data on admission to hospital, without the need for specific feature selection or missing data imputation. CovRNN was designed to predict three outcomes: in-hospital mortality, need for mechanical ventilation, and prolonged hospital stay (>7 days). For in-hospital mortality and mechanical ventilation, CovRNN produced time-to-event risk scores (survival prediction; evaluated by the concordance index) and all-time risk scores (binary prediction; area under the receiver operating characteristic curve [AUROC] was the main metric); we only trained a binary classification model for prolonged hospital stay. For binary classification tasks, we compared CovRNN against traditional machine learning algorithms: logistic regression and light gradient boost machine. Our models were trained and validated on the heterogeneous, deidentified data of 247 960 patients with COVID-19 from 87 US health-care systems derived from the Cerner Real-World COVID-19 Q3 Dataset up to September 2020. We held out the data of 4175 patients from two hospitals for external validation. The remaining 243 785 patients from the 85 health systems were grouped into training (n=170 626), validation (n=24 378), and multi-hospital test (n=48 781) sets. Model performance was evaluated in the multi-hospital test set. The transferability of CovRNN was externally validated by use of deidentified data from 36 140 patients derived from the US-based Optum deidentified COVID-19 electronic health record dataset (version 1015; from January, 2007, to Oct 15, 2020). Exact dates of data extraction were masked by the databases to ensure patient data safety. FINDINGS: CovRNN binary models achieved AUROCs of 93·0% (95% CI 92·6-93·4) for the prediction of in-hospital mortality, 92·9% (92·6-93·2) for the prediction of mechanical ventilation, and 86·5% (86·2-86·9) for the prediction of a prolonged hospital stay, outperforming light gradient boost machine and logistic regression algorithms. External validation confirmed AUROCs in similar ranges (91·3-97·0% for in-hospital mortality prediction, 91·5-96·0% for the prediction of mechanical ventilation, and 81·0-88·3% for the prediction of prolonged hospital stay). For survival prediction, CovRNN achieved a concordance index of 86·0% (95% CI 85·1-86·9) for in-hospital mortality and 92·6% (92·2-93·0) for mechanical ventilation. INTERPRETATION: Trained on a large, heterogeneous, real-world dataset, our CovRNN models showed high prediction accuracy and transferability through consistently good performances on multiple external datasets. Our results show the feasibility of a COVID-19 predictive model that delivers high accuracy without the need for complex feature engineering. FUNDING: Cancer Prevention and Research Institute of Texas.

Real World Long-term Assessment of The Efficacy of Tocilizumab in Patients with COVID-19: Results From A Large De-identified Multicenter Electronic Health Record Dataset in the United States.

BACKGROUND: Studies have shown conflicting results on the efficacy of tocilizumab (TCZ) for patients with COVID-19, with many confounders of clinical status and limited duration of the observation. Here, we evaluate the real-world long-term efficacy of TCZ in COVID-19 patients. METHODS: We conducted a retrospective study of hospitalized adult patients with COVID-19 using a large US-based multicenter COVID-19 database (Cerner Real-World Data; updated in September, 2020). The TCZ group was defined as patients who received at least one dose of the drug. Matching weight (MW) and a propensity score weighting method were used to balance confounding factors. RESULTS: A total of 20,399 patients were identified. 1,510 and 18,899 were in the TCZ and control groups, respectively. After MW adjustment, no statistically significant differences in all-cause mortality were found for the TCZ vs. control group (Hazard Ratio [HR]:0.76, p=0.06). Survival curves suggested a better trend in short-term observation, driven from a subgroup of patients requiring oxygen masks, BIPAP or CPAP. CONCLUSION: We observed a temporal (early) benefit of TCZ, especially in patients on non-invasive high-flow supplemental oxygen. However, the benefit effects faded with longer observation. The long-term benefits and risks of TCZ should be carefully evaluated with follow-up studies.